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1. A method of displaying an output image of u scene from a freely 
selectable viewpoint, the method comprising the steps of 

obtaining input images of the scene, each from a respective input viewpoint; 
computing modeLinformation representing points located in a space of the scene 
5 from which input information in the input images originates; 
selecting the selectable viewpoint; 

determining for an output pixel in the output image which output point 
represented in the model information is visible in the output image at that output pixel; 

determining a pixel value of that output pixel from the image information in the 
10 input image or input images which originated from the output point; 

displaying the output pixel in the output image according to the pixel value, 

-char act e rized in that the method compri s es 

computing a respective model for each input image, the respective model 
comprising information about surface patches located in the space of the scene, substantially 
15 each surface patch corresponding to a respective set of pixels in the respective input image; 

determining for each respective model which, if any, respective point from the 
surface patches of that respective model is visible in the output image at the output pixel 
according to that respective model, 

selecting the output point from the respective, points on the basis of comparison 
20 of parameters of the surface patches containing the respective points in different models for 
which the respective points have been found; 

determining the pixel value from the image information in the input image 
corresponding to the respective model from which the output point is selected. 

2. A method according to Claim 1, wherein for selecting the output point 
25 from the respective points, preference is given to respective points with smaller distance to 

the selected viewpoint. 

3. A method according to Claim 1, wherein for selecting the output point 
from the respective points, preference is given to respective points with smaller difference 
between a first and second angle between a normal of the surface patch to which the 
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respective point belongs and lines of sight to the respective point, from the selected 
viewpoint and the respective viewpoint of the input image corresponding to the respective 
point respectively. 

4. A method according to Claim 1 - characterized in that , for selecting the 

5 output point, preference is given to respective points with smaller difference between a first 
and second size of a projection of the output pixel in the output image and an input pi;.el in 
the input image corresponding to the respective point respectively, when the output pixel and 
the input pixel are back-projected onto the surface patch. 

5. A method according to Claim 1, wherein for each particular input image 
10 an ordered list of intervals in the output image is computed, the intervals corresponding to 

segments of respective surface patches successively visible along a scan-line in the output 

image, the scan-line containing the output pixel, the output point being determined by 
5 determining the interval in which the output pixel lies, and wherein the ordered list is 
Q computed by processing the line-segments successively in an order in which they are visible 

sj 15 along a path in the particular input image, a representation being kept of a continuous range 
'fl along the scan-line spanned by the intervals of processed line-segments, a 

O foreground/background position of a line-segment being determined by comparing the 

t L line-segment to an auxiliary line segment extending between points in the model 

corresponding to the endpoints of the continuous range. 
m 20 6. A method according to Claim 1, wherein altered pixel values are 

JSi computed for pixels of the input images, so as to represent the effect of lighting changes, the 

method comprising 

selecting a source point in the scene and a lighting distribution image located 
relative to the source point, a respective lighting pixel value being associated with each light 
25 pixel in the lighting distribution image; 

determining for each respective model which, if any, respective lighting point 
from the surface patches of that respective model would be visible in the lighting distribution 
image at a light pixel according to that respective model, 

selecting an alterable point from the respective lighting points on the basis of 
30 comparison of parameters of the surface patches containing the respective lighting points in 
different models for which the respective lighting points have been found; 

altering the image information in the input image corresponding to the 
respective model from which the alterable point is selected at an alterable pixel 
corresponding to the alterable point. 
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7. A method according to Claim 1 comprising 

obtaining further input images from a further scene, each from a respective 
further viewpoint in combination with a respective further models comprising information 
about surface patches located in a space of the further scene, substantially each surface patch 
corresponding to a respective set of pixels in the further input image; 

designating a collection of further surface patches from the further models; 
selecting a position and attitude for the collection of further surface patches 




relative to the scene; 

- r I determining respective auxiliary viewpoints relative to the further scene so that 
CL^ 10 each respective auxiliary ^Viewpoint- has a same spatial relation to the further surface patches 

as a respective one of the viewpoints when the further surface patches are located according 

to the selected position and attitude; 
O - computing a set of auxiliary images of the further surfaces in the collection 

5 from the auxiliary viewpoints and computing auxiliary models describing the further surface 

SJj 15 patches in the collection as far as they correspond to sets of pixels in the auxiliary images; 
sO - replacing a pixel value and model information in the input images by a pixel 

h value and auxiliary model information from the auxiliary images prior to computing the 

% output image when a depth from the respective viewpoint of the input image to a point in the 

4? scene represented by the pixel value is greater than a depth from the corresponding auxiliary 

m 20 viewpoint to a visible point on a further surface patch from the collection. 
* 8. A device for displaying an output image of a scene from a freely 

selectable viewpoint, the device comprising 

memory for storing input images of the scene, each from a respective input 

viewpoint; 

25 - memory for storing model information representing points in the scene from 

which input information in the input images originates; 

means for selecting the selectable viewpoint; 

means for determining for an output pixel in the output image which output 
point represented in the model information is visible in the output image at that output pixel; 
30 - means for determining a pixel value of that output pixel from the image 

information in the input image or images which originated from the output point; 

a display unit receiving the pixel value for displaying the output pixel in the 
output image according to the pixel value, 

- eharactcriz e d in thafr the memory for storing model information is arranged to store a 
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respective model for each input image, the respective model comprising information about 
surface patches located in the space of the scene, substantially each surface patch 
corresponding to a respective set of pixels in the respective input image; the device 
comprising 

5 - means for determining for each respective model which, if any, respective point 

from the surface patches of that respective model is visible in the output image at the output 
pixel according to that respective model, 

means for selecting the output point from the respective points on the basis of 
comparison of parameters of the surface patches containing the respective points in different 
10 models for which the respective point has been found; 

means for determining the pixel value from the image information in the input 
image corresponding to the respective model from which the output point is selected. 

9. A device according to Claim 8 wherein altered pixel values are computed 
for pixels of the input images, so as to represent the effect of lighting changes, the device 

15 comprising 

means for selecting a source point in the scene and a lighting distribution image 
located relative to the source point, a respective lighting pixel value being associated with 
each light pixel in the lighting distribution image; 

means for determining for each respective model which, if any, respective 
20 lighting point from the surface patches of that respective model would be visible in the 
lighting distribution image at a light pixel according to that respective model, 

means for selecting an alterable point from the respective lighting points on the 
basis of comparison of parameters of the surface patches containing the respective lighting 
points in different models for which the respective lighting points have been found; 
25 - means for altering the image information according to a lighting model in the 

input image corresponding to the respective model from which the alterable point is selected. 

10. A device according to Claim 8, wherein the means for determining for 
each model which output point is visible determine for each particular input image an 
ordered list of intervals in the output image is computed, the intervals corresponding to 

30 segments of respective surface patches successively visible along a scan-line in the output 
image, the scan-line containing the output pixel, the output point being determined by 
determining the interval in which the output pixel lies, and wherein the ordered list is 
computed by processing the line-segments successively in an order in which they are visible 
along a path in the particular input image, a representation being kept of a continuous range 
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along the scan-line spanned by the intervals of processed line-segments, the 
foreground/background position of a line-segment being determined by comparing the 
line-segment to an auxiliary line segment extending between points in the model 
corresponding to the endpoints of the continuous range. 
11. A device according to Claim 8, comprising means for 

obtaining further input images from a further scene, each from a respective 
further viewpoint; 

designating a collection of further surface patches from the further models; 
n selecting a position and attitude for the collection of further surface patches 



10 relative to the scene; 



- r r determining respective auxiliary viewpoints relative to the further scene so that 
each respective auxiliary .- viewpoint has a same spatial relation to the further surface patches 



as a respective one of the viewpoints when the further surface patches are located according 
ej to the selected position and attitude; 

Jf : 15 - computing a set of auxiliary images of the further surfaces in the collection 

MS from the auxiliary viewpoints and computing auxiliary model information representing depth 

b of the further surface patches in the collection from the auxiliary viewpoints; 

L. - replacing a pixel values and model information in the input images by a pixel 

-P value and auxiliary model information from the auxiliary images prior to computing the 

m 20 output image when a depth from the respective viewpoint of the input image to a point in the 
scene represented by the pixel value is greater than a depth from the corresponding auxiliary 
viewpoint to a visible point on a further surface patch from the collection. 



